Repository navigation
A run writes its genotype truth beside its records - #49
Merged
Merged
Conversation
`Genotype.to_table()` is one row per gene with the alleles as list-valued columns. That shape is wrong for a TSV a benchmark joins against: a list inside a flat cell is a nested format inside a flat one, and every consumer writes its own splitter. `genotype_truth_rows` projects it into one row per (gene, chromosome, carried allele). `to_table()` is unchanged -- it is a public, tested surface with its own callers, and this is a projection of it, not a replacement. Two properties are pinned harder than the shape. A deleted gene still gets rows. It is the one point where the long form can lose what the wide form keeps: "absent from the table" and "present and deleted" are different truths, and a genotype caller is scored on exactly that difference. Deleted on one chromosome gives one allele row and one `(none)` row; deleted on both gives two. `gene_state` carries the zygosity verbatim so the wide form stays recoverable. The allele cell for a deleted chromosome is the literal token `(none)`, not an empty cell, because an empty TSV cell cannot be told apart from a missing value. `locus_of` is a deliberate port of the engine's `locus_prefix` from engine_rs/src/airr_record/locus.rs: the uppercased three-character prefix, accepted only for the seven AIRR loci, empty otherwise. The sidecar is joined against a records file, and a reader will compare `locus` across the two; a rule that drifted from the engine's would put a value in the truth file that the records disagree with. A synthetic reference gets an empty locus in both places. An unplaced novel allele contributes no row: defined but never assigned to a chromosome, so not carried, so not truth about this genome.
`to_genotype_tsv` on SimulationResult and CohortResult. A records file carries `subject_id` and nothing that says what that subject's genome was; this is the companion artifact that closes the join. The refusals carry more weight here than the happy path. A result with no genotype raises instead of writing a header-only file, because a zero-row truth file beside a million records is the failure mode that looks like success -- the export appears to have worked and the benchmark silently scores against nothing. A genotype with no subject_id raises for the same reason: the sidecar would be unjoinable while looking complete. Both checks run before the file is opened, so a refused export leaves nothing behind rather than a partial artifact someone later mistakes for truth. Writes with an explicit encoding, matching `to_fasta` rather than the platform default.
…decar The measurement that justified this phase was a probe run once by hand: over a 2-subject cohort, 0 of 183 distinct V/D/J calls were missing from the genotype table. This turns it into a standing guarantee over a 3-subject, 450-record cohort. A sidecar that misses an allele is worse than no sidecar: a benchmark scoring a caller against it counts a correct call as wrong. Every assertion is paired with a guard on its own premise, because a completeness test runs green on a degenerate repertoire -- one subject, one allele per segment, no deletions. The guards assert that the run produced records for all three subjects, that more than one allele was drawn per segment, that the three genotypes differ from each other, that at least one subject carries a deleted gene, and that no subject carries the pooled allele set (which would make per-subject truth indistinguishable from pooled truth). The deletion is forced rather than hoped for. Whether `Genotype.sample` happens to delete a gene at these seeds is not something the test should depend on. The novel-allele test makes its gene the subject's only V gene. Every V gene in this cartridge carries the same usage weight, so drawing one specific gene is a ~0.3% event per record: measured, 10 hits in 3000 records. A test that hoped for the draw would need thousands of records and still flake. Verified non-vacuous against three injected defects, each caught by the test written for it: dropping chromosome 1 fails completeness; never flagging novel fails the novel test; omitting deleted rows fails both the deletion premise and the deleted-genes test.
`germline.genotype` carried a truth_gap stating that the genotype "is not exported" and that there is "no haplotype or chromosome field". The second half had never been true: the engine stamps `haplotype` -- the chromosome the rearrangement drew from -- on every record of a genotype run, and `subject_id` beside it. Nine existing tests read `record["haplotype"]` while the catalog said the field did not exist. The reason the contradiction survived is the field census. No mode in `record_modes` attached a genotype, so `subject_id` and `haplotype` were invisible to `test_no_exported_field_is_owned_by_nobody` and to `test_every_truth_field_is_a_field_some_mode_actually_produces` -- the same blind spot the duplex mode was added to fix, and the module's own docstring says so about that one. So the census gains a genotype mode first. It immediately reported two exported fields nobody is answerable for, which is the check doing its job on a defect it had been unable to see. `germline.genotype` now claims all three of its fields, the gap is gone (the carried-allele set is exported by `to_genotype_tsv` as of this branch), and what replaces it is a note saying where that truth lives and why per-read novelty is a join rather than a column. `haplotype` is verified on a reference basis by the phasing test, which reads the drawn chromosome off a genotype it constructed itself. That test also gains a premise guard: without it the claim held in one direction only, since a run that never left chromosome 0 satisfies every assertion in it while proving nothing about chromosome 1. `locus` and `subject_id` are excluded from the denominator and pinned as such -- both are echoed onto every record rather than sampled per record -- so one field carries the row. The coverage pin moves deliberately: verified 22 -> 23, no-truth-exported 4 -> 3. It is the second row to leave `no-truth-exported` by the gap closing rather than by a test being added, and the first where part of the gap was simply wrong.
The genotype guide gains `to_genotype_tsv` beside `Genotype.to_tsv`: the long form, one row per carried allele, keyed on `subject_id` so it joins to the records, with the `(none)` sentinel for a deleted chromosome named where a reader will meet it. The benchmarking recipe writes it alongside the AIRR table, since a dataset handed to someone else should arrive with the truth it is scored against. There is deliberately no per-read novelty column, so the join is shown concretely rather than described: read the sidecar's novel pairs into a set, then test `(subject_id, truth_v_call)` against it. A reader who has to invent that will add the column themselves, which is exactly what the record schema is trying to avoid. The same join answers the deletion and zygosity questions per read. Run end to end before being written down: 27 of 50 records when the planted novel allele is the subject's only V option, 0 when it is one of 52, which is the right answer in both directions.
…side A whole-branch review found the sidecar writing a truth file that is a silent subset of the truth, which is the one failure this module was written to prevent. `Genotype.permissive` leaves unspecified genes to be sampled over the whole reference, unphased, and `to_table` emits a row only per *specified* gene. So a permissive genotype projected to a sidecar loses every gene nobody pinned. Measured on HUMAN_IGH_OGRDB: `permissive().with_subject(...)` wrote a 66-byte header-only file and returned successfully while the records named five real alleles; with one gene pinned it wrote two rows while the records drew 119 alleles absent from the file. A benchmark scoring a caller against that marks correct calls wrong, which is worse than having no file at all. The writer now refuses, keyed on what is specified rather than on `is_permissive`, because `permissive().complete_from_reference()` stays permissive while its table becomes complete. One existing test was built on a permissive genotype and so was asserting the sentinel inside an already incomplete table; it is rebuilt on a complete one. Two neighbouring refusals for the same reason: a cohort where only some subjects carry a genotype used to drop the Nones and write a table short by whole subjects, and two subjects sharing one `subject_id` used to merge under one join key. The `locus` column's documented property was false. A record carries one locus for the whole rearrangement, taken from its V call; a row carries the locus of its own gene. On HUMAN_TCRD_IMGT, whose V pool holds both TRAV and TRDV genes, 134 of 200 records disagree with the sidecar row for the very allele they were generated from. The join is on the allele and is unaffected, so the column is now documented as what it is, with a warning against filtering by it, and the test class that was named for the records -- and never looked at one, which is how this survived -- now reads real records on both a single-locus and a mixed-locus cartridge. `germline.genotype` keeps a truth_gap, in reduced form. Removing it outright over-claimed: per-chromosome expression weight is exported nowhere, so a genotype with weights (1.0, 0.0) projects 85 rows claiming haplotype-1 alleles are carried while no record can draw one. That is the remaining part of the mechanism's own summary clause. The pin moves to gap-limited, so the distribution is 22 verified / 4 gap-limited / 3 no-truth / 2 consistent. Also: the sidecar wrote CRLF beside a records file that pins LF, leaving a stray carriage return on the last column in any shell pipeline joining the two; four of the eight columns had no file-level assertion at all, including the `novel` column whose spelling the documented join compares against the literal "True"; the deletion question was documented as answerable per read, which it is not, since a deleted gene is never drawn from; the fully-deleted premise guard was satisfied by a hemizygous deletion, which would have left its test asserting nothing; and `Genotype.to_tsv` opened without an explicit encoding, so a non-ASCII subject_id raised on Windows.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A simulated dataset used to leave this library through two doors: the records
from the result, the genotype from the
Genotypeobject the caller still hadto be holding. Hand the dataset to someone else and it arrived without the
thing it was meant to be scored against. This adds the sidecar that closes
that, and corrects a test-suite catalogue that described the area wrongly.
What is new
result.to_genotype_tsv(path)andCohortResult.to_genotype_tsv(path)writethe genotype as one row per carried allele —
subject_id,locus,segment,gene,allele,chromosome,novel,gene_state— so one sidecar joinsto one records file.
Genotype.to_table()/to_tsv()are untouched; this isa projection of them, not a replacement.
Two decisions worth stating:
(none). "Geneabsent from the table" and "gene present and deleted on both chromosomes"
are different truths, and a genotype caller is scored on exactly that
difference. An empty TSV cell cannot be told apart from a missing value.
of
truth_v_callagainst the sidecar on(subject_id, allele). The recordalready carries both halves, so a derived column would be a third copy. The
guide shows the join concretely, run end to end before being written down:
27 of 50 records when the planted novel allele is the subject's only V
option, 0 when it is one of 52.
What it refuses, and why that is most of the work
A zero-row truth file beside a million records is a failure that looks like
success, so the writer refuses rather than writing one. Four cases, each
measured:
engine samples those genes over the whole reference, unphased, so their
alleles reach the records but no row of the file.
permissive()alone wrotea 66-byte header-only file while the records named five real alleles; with
one gene pinned it wrote two rows while the records drew 119 alleles
absent from the file. Keyed on what is specified, not on
is_permissive,because
permissive().complete_from_reference()stays permissive while itstable becomes complete.
be dropped, producing a table short by whole subjects that looked complete.
subject_id— their rows would merge under onejoin key and neither subject's truth would be recoverable.
Every refusal happens before the file is opened, so a refusal cannot leave a
partial artifact behind.
The catalogue was wrong, not just stale
germline.genotypedeclared that the genotype "is not exported" and thatthere is "no haplotype or chromosome field". The second half had never been
true:
haplotypeis recorded by the engine and stamped on every record of agenotype run, and nine existing tests read it while the catalogue said the
field did not exist.
The reason it survived is the field census — no mode in
record_modesattached a genotype, so
subject_idandhaplotypewere invisible to the twocompleteness checks that exist to catch exactly this. That module's own
docstring already describes the same failure for the duplex mode. Adding the
genotype mode immediately reported two exported fields nobody was answerable
for.
The gap is narrowed, not removed. Per-chromosome expression weight is
exported nowhere, so a genotype with
chromosome_weights(1.0, 0.0)projects85 rows claiming haplotype-1 alleles are carried while no record can draw one.
Deleting the gap outright would have over-claimed.
germline.genotypemovesno-truth-exported→gap-limited; the pin is now 22 verified / 4gap-limited / 3 no-truth / 2 consistent.
haplotypeis covered on a reference basis by the phasing test, which readsthe drawn chromosome off a genotype it constructed itself. That test also
gained a premise guard: without it the claim held in one direction only, since
a run that never left chromosome 0 satisfies every assertion in it while
proving nothing about chromosome 1.
One documented property was false
A record carries one
locusfor the whole rearrangement, taken from its Vcall. A sidecar row carries the locus of its own gene. On
HUMAN_TCRD_IMGT,whose V pool holds both
TRAVandTRDVgenes, 134 of 200 recordsdisagree with the sidecar row for the very allele they were generated from.
The join is on the allele and is unaffected, so the column is now documented
as what it is, with a warning against filtering by it. The test class that was
named for this property — and never looked at a record, which is how it
survived — now reads real records on both a single-locus and a mixed-locus
cartridge.
Verification
pytest -n auto -m "not performance and not campaign": 5856 passed, 17skipped (5812 before the branch)
pytest -m performance, serial: 9 passedpytest tests/acceptance/: 94 passed — the sha256 determinism andreproducibility pins are green untouched. No engine or record bytes
change; the two new methods are called by nothing in a run path.
cargo test --all-features: 1735 passed across 6 suites (noengine_rs/file is touched)
ruff check tests/ src/ tools/: cleaninventory_gate.py check --since origin/master --declarations: cleanThe completeness suite was proved non-vacuous against three injected defects:
dropping chromosome 1 fails completeness, never flagging
novelfails thenovel test, and omitting deleted rows fails both the deletion premise and the
deleted-genes test.
Also in here
The sidecar wrote CRLF beside a records file that pins LF, leaving a stray
carriage return on the last column in any shell pipeline joining the two. Four
of the eight columns had no file-level assertion, including
novel, whosespelling the documented join compares against the literal
"True". Thedeletion question was documented as answerable per read, which it is not,
since a deleted gene is never drawn from. The fully-deleted premise guard was
satisfied by a hemizygous deletion, which would have left its test asserting
nothing. And
Genotype.to_tsvopened without an explicit encoding, so anon-ASCII
subject_idraised on Windows — pre-existing, but the guide nowrecommends it beside the new method.